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Countering  Positive  Confirmation  Biases  in  Command  Teams:  An  Experiment  with  Different 

Interventions 

Abstract 

As  part  of  the  larger  sensemaking  effort  in  the  Singapore  Armed  Forces  (SAF),  the  Singapore  Armed 
Forces  Centre  for  Military  Experimentation  (SCME)  continuously  strives  to  challenge  our  C2  status  quo. 
This  has  generated  a  number  of  C2  ideas  that  are  currently  undergoing  experimentation  within  SCME. 

One  such  idea  is  the  Command  Post  Anywhere  (CPA)  concept  which  experiments  with  the  idea  of  a 
highly  survivable  future  command  post  capable  of  full  distributed  collaboration  in  a  networked 
environment.  In  the  middle  of  last  year,  SCME  went  further  and  posed  herself  the  question  of  whether 
other  useful  concepts  could  be  built  on  top  the  fundamental  idea  of  distributed  collaboration.  This  led  to  a 
research  in  psychological/technological  and  social  concepts  such  as  groupware  awareness,  groupthink, 
insight/creativity  and  decision  making  biases.  Decision  making  biases,  in  particular  positive  confirmation 
bias,  would  eventually  become  the  focus  of  the  experiment  conducted  in  October  2004.  For  the  military, 
there  could  be  dire  consequences  as  a  result  of  positive  confirmation  bias.  It  could  leniently  mean  a  need 
to  re-plan  or  the  catastrophic  loss  of  units  at  the  other  extreme.  In  this  paper,  the  question  of  how  positive 
confirmation  bias  could  be  addressed  and  mitigated  in  a  command  team  setting  with  different  C2 
interventions  was  investigated. 
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Introduction 


As  part  of  the  larger  3rd  Generation  SAF  sensemaking  effort,  SCME  continuously  strives  to  challenge  our 
C2  status  quo.  This  endeavor  has  generated  a  number  of  new  C2  ideas  that  are  currently  undergoing 
experimentation.  One  such  idea  is  the  Command  Post  Anywhere  (CPA)  concept  which  experiments  with 
the  concept  of  a  highly  survivable  future  command  post  capable  of  full  distributed  collaboration  in  a 
networked  environment.  In  July  04,  SCME  went  further  and  posed  herself  the  question  of  whether  other 
C2  concepts  could  be  built  upon  the  fundamental  idea  of  distributed  collaboration.  Several  areas  were 
looked  at  in  the  ensuing  research.  In  chronological  order,  they  were:  group  awareness,  insight/creativity, 
groupthink  and  decision  making  biases. 

The  work  of  Gutwin  (1997),  Ellis,  Gibbs,  &  Rein  (1991),  Gutwin,  Roseman,  &  Greenberg  (1996), 

Lococo  &  Yen  (1998),  Migliarese  &  Paolucci  (1995),  Pendergast  &  Hayne  (1999)  and  Poltrock  & 
Engelbeck  (1999)  were  reviewed  under  groupware  awareness.  Although  research  in  this  area  appears  to 
be  very  interesting,  they  are  deemed  more  suitable  and  applicable  to  SCME’s  experimentation  in 
distributed  collaboration.  It  will  likely  do  less  in  facilitating  SCME’s  desire  to  break  out  into  uncharted 
C2  territory.  For  this  reason,  groupware  awareness  was  left  to  be  pursued  by  SCME’s  experimenters 
working  on  distributed  collaboration. 

Under  insight  and  creativity,  the  work  of  Gilson  &  Shalley  (2004),  Mayer  (1995),  Seifert,  Meyer, 
Davidson,  Patalano,  &  Ilan  (1995),  Schooler,  Fallshore,  &  Fiore  (1995),  Dunbar  (1995),  Davidson  (1995) 
were  reviewed.  There  is  a  lack  of  evidence  to  show  that  insight  and  creativity  could  be  generated  on  a 
consistent  basis.  Further,  there  also  seems  to  be  a  lack  of  consensus  on  the  metaphor  that  could  illuminate 
all  facets  of  insights.  Justifiably,  the  interest  to  experiment  and  develop  C2  interventions  for  insight  and 
creativity  waned.  These  two  areas  were  however  flagged  as  watch  areas  so  that  SCME  could  harvest 
successful  research  in  the  future. 

In  groupthink,  the  work  of  McCauley  (1998),  Hogg  &  Hains  (1998),  Park  (2000),  Raven  (1998),  Turner 
&  Pratkanis  (1998b),  Fuller  (1998),  Hart  (1998),  Whyte  (1998),  Moorhead,  Neck,  &  West  (1998),  Esser 
(1998),  Kramer  (1998),  Turner  &  Pratkanis  (1998a)  and  Peterson,  Owens,  Tetlock,  Fan,  &  Martorana 
(1998)  were  reviewed.  As  a  concept,  groupthink  appears  weaker  than  all  previous  concepts  that  were 
researched  because  there  is  still  a  controversy  that  surrounds  its  existence  even  as  a  phenomenon.  It  was 
therefore  decided  that  this  may  not  be  a  fruitful  area  for  further  exploration  in  the  immediate  future. 

After  rounding  off  the  research  on  Groupthink,  there  was  an  inkling  that  past  research  in  the  area  of 
decision  making  might  potentially  be  useful.  A  research  approach  was  conceived  to  identify  those 
psychological  phenomena  that  have  been  found  to  work  against  the  estimate  of  “truth”  and  see  if  C2 
interventions  could  be  developed  to  counter  them.  In  warfare  addressing  this  issue  is  still  pertinent  even  in 
the  age  of  network  centricity  because  it  has  been  recognized  that  the  fog  of  war  whether  created  by  the 
enemy  or  by  your  own  forces  will  continue  to  exist.  Wickens  &  Hollands  (2000)  identified  three  of  such 
phenomena  in  overconfidence  bias,  anchoring  heuristic  and  positive  confirmation  bias.  All  three  of  them 
could  work  against  the  effort  to  gather  a  true  picture  for  the  puipose  of  problem  solving.  As  a  result  of 
this  research,  these  have  been  deemed  to  be  potentially  useful  areas  for  future  research  and 
experimentation  in  SCME  along  with  other  “cognitive  biases”  that  are  found  in  the  literature  (  Kahneman, 
Slovic,  &  Tversky  (1982)  and  Fischoff  (1988) ). 

The  experiment  in  this  paper  describes  SCME’s  nascent  effort  in  addressing  one  of  them;  positive 
confirmation  bias. 
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Literature  Review 


What  is  Positive  Confirmation  Bias? 


Positive  confirmation  bias  is  the  tendency,  that  have  been  observed  in  many  situations,  for  people  to  seek 
information  and  cues  to  confirm  the  tentatively  held  hypothesis,  rather  than  seek  information  to  disprove 
them  (Wickens  &  Hollands,  2000).  It  has  also  been  found  (  Einhorn  &  Hogarth  (1978),  Mynatt,  Doherty, 
&  Tweney  (1977),  Schustack  &  Sternberg  (1981)  and  Wason  &  Laird,  1972  )  that  there  are  high 
likelihoods  that  cues  whether  certain  or  uncertain  will  be  interpreted  in  a  manner  that  supports  the 
tentatively  held  hypothesis.  This  then  produces  a  “cognitive  tunnel  vision”  (  S.  D.  Woods,  Johanensen, 
Cook,  &  Starter  (1994),  D.  Woods  &  Cook  (1999) ),  in  which  there  will  be  failure  to  encode  or  process 
information  that  is  contradictory  to  or  inconsistent  with  the  tentatively  held  hypothesis.  This  “cognitive 
tunnel  vision”  can  be  disastrous  during  warfare,  especially  when  situations  are  complex  or  even  chaotic. 
There  could  also  be  other  dire  consequences  as  a  result  of  positive  confirmation  biasness;  notably  when 
the  hypothesis  is  wrong.  It  could  leniently  mean  a  need  to  re -plan  or  the  catastrophic  loss  of  units  at  the 
other  extreme;  extremes  that  no  military  can  afford. 

Examples  of  Confirmation  Bias  leading  to  Catastrophes 

There  are  many  examples  in  history  where  positive  confirmation  bias  has  resulted  in  the  wrong  decisions. 
The  author  highlights  two  of  them  found  in  Wickens  &  Hollands  (2000).  They  are:  (1)  the  USS 
Vincennes  incident  in  which  an  Iranian  commercial  airliner  was  erroneously  shot  down  over  the  Persian 
Gulf  in  1988  and  (2)  the  1979  Thee  Mile  Island  incident  in  which  led  to  a  nuclear  reactor  meltdown.  For 
more  information  on  these  two  incidents,  the  interested  reader  can  refer  to  Klein  (1996)  and  U.S.  Nuclear 
Regulatory  Commission  (NRC)  (2005). 

The  investigations  into  the  USS  Vincennes  incident  revealed  that  the  tentatively  held  hypothesis  of  the 
approaching  aircraft  being  hostile  was  erroneous  (U.S.  Navy,  1988).  The  captain  seemingly  failed  to  take 
into  account  the  alternate  hypothesis  that  the  aircraft  could  have  been  a  commercial  aircraft  even  though 
that  alternate  hypothesis  was  floated  on  more  than  one  occasions  in  the  minds  of  the  crew  from  its  initial 
detection  until  its  final  engagement.  During  the  investigation,  it  was  also  found  that  a  visual  identification 
option  through  the  vectoring  of  two  F-14s  on  Combat  Air  Patrol  (CAP)  to  the  aircraft  in  question  was  not 
exercised.  Through  extrapolation  of  the  positive  confirmation  bias  explanation,  it  is  imaginable  that  the 
instruction  to  the  two  pilots  would  have  been  “Confirm  Hostile”  instead  of  “Confirm  Friendly”. 

The  investigations  into  the  Three  Mile  Island  nuclear  power  incident  also  revealed  the  existence  of 
positive  confirmation  bias  in  play.  On  the  fateful  day  of  March  28,  1979,  the  crew  at  the  Three  Mile 
Island  nuclear  plant  received  an  alert  of  a  plant  failure  at  0400  hours.  They  focused  their  attention  on  a 
display  that  erroneously  indicated  that  a  relief  valve  had  closed  and  held  the  tentative  hypothesis  that  the 
water  level  was  too  high  in  the  plant  and  was  approaching  danger  levels.  The  crew  then  chose  an  incorrect 
action  to  override  an  emergency  relief  pump  that  provided  cooling  water  to  the  reaction.  The  truth  was 
that  the  water  level  was  actually  dangerously  low  and  was  in  the  danger  causing  a  reactor  meltdown. 
According  to  the  investigation,  the  crew  apparently  did  not  attend  to  the  many  other  cues  indicating  low 
water  level  but  instead  counter-interpret  them  as  evidence  for  the  high  water  hypothesis  (Rubenstein  & 
Mason,  1979).  The  accident  at  the  Three  Mile  Island  was  the  most  serious  in  U.S.  commercial  nuclear 
power  plant  operating  history.  Even  though  it  led  to  no  deaths  or  injuries,  it  brought  about  sweeping 
changes  involving  emergency  response  planning,  reactor  operator  training,  human  factors  engineering, 
radiation  protection,  and  many  other  areas  of  nuclear  power  plant  operations. 

It  should  be  apparent  that  if  positive  confirmation  bias  can  exist  in  a  well  trained  and  tightly  knitted  unit 
such  as  that  in  the  USS  Vincennes  Combat  Information  Center  (CIC)  and  nuclear  power  plant,  the 
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possibility  of  it  occurring  among  combat  teams  of  the  future  where  they  are  going  to  be  highly 
composable  and  more  modular  remains  high.  The  seriousness  of  its  repercussions  as  illustrated  by  the 
previous  examples  therefore  warrants  its  deeper  examination  as  a  phenomenon  and  more  importantly  how 
it  could  be  addressed  with  training  and  C2  interventions. 

What  is  the  Issue  in  Addressing  Confirmation  Bias? 

According  to  Klayman  &  Ha  (1987),  there  is  some  evidence  that  the  confirmation  bias  is  not  altogether  a 
bad  strategy.  At  the  very  least,  maintaining  a  “working  hypothesis”  is  valuable  because  it  provides  a  guide 
to  the  search  for  new  information  and  this  is  more  efficient  than  a  random  search. 

The  issue  therefore  is  not  one  where  nobody  should  hold  any  hypothesis  about  the  situation  but  rather 
how  to  force  the  simultaneous  entertainment  of  alternative  hypotheses  and  to  seek  disconfirming 
evidence,  or  at  least  attend  to  it,  if  it  arrives. 

Given  that  our  future  forces  will  likely  fight  in  more  modular  units  and  in  a  more  plug  and  play  manner,  it 
was  also  clear  that  the  issue  of  positive  confirmation  bias  needs  to  be  addressed  not  in  an  individual  but  a 
team  setting. 

A  series  of  questions  regarding  the  actual  results  of  past  positive  confirmation  studies  conducted  and  their 
methodologies  used  surfaced.  The  guiding  question  in  this  part  of  the  research  was  whether  there  are 
results  from  past  research  that  could  be  consumed  by  SCME  directly.  Through  further  research,  an 
opinion  was  formed  that  dedicated  experimentation  on  positive  confirmation  bias  had  to  be  conducted  for 
SCME  to  move  on. 

Results  and  Methods  of  Past  Research  in  Positive  Confirmation  Bias 


The  original  selection  task  used  for  positive  confirmation  bias  research  was  developed  by  Wason  (1968b) 
and  Wason  &  Laird  (1972).  A  typical  experiment  uses  a  layout  of  four  double-sided  cards.  Subjects  are 
told  that  each  card  has  a  letter  on  one  side  and  a  number  on  the  other,  but  they  can  see  only  the  upper 
faces  of  the  four  cards.  These  show  ‘A’,  ‘D’,  ‘4’  and  ‘7’.  Each  subject  is  asked  to  consider  the  following 
‘rule’,  as  applied  to  the  four  cards:  ‘If  a  card  has  a  vowel  on  one  side,  then  it  has  an  even  number  on  the 
other  side’.  The  instruction  takes  the  form:  ‘Your  task  is  to  say  which  of  the  cards  you  need  to  turn  over  to 
find  out  whether  the  rule  is  true  or  false.’ 

The  two  most  common  responses  are  the  ‘A’  card  alone,  and  the  ‘A’  and  ‘4’  cards  in  combination.  The 
correct  answer  to  the  question  posed  is  the  combination  of  ‘A’  and  ‘7’.  This  is  because  the  frequently 
chosen  ‘4’  card  can  provide  no  information  relevant  to  the  issue  of  whether  the  rule  is  true  or  false.  Notice 
however,  that  the  ‘A’  and  ‘4’  cards  are  the  ones  that  are  capable  of  providing  evidence  which  confirms 
the  rule,  the  ‘7’  card  can  only  disconfirm  the  rule  (i.e.  by  revealing  a  card  which  has  a  vowel  on  one  side 
but  not  an  even  number  on  the  other).  According  to  Wason,  the  evidence  from  the  selection  task  can  be 
interpreted  as  consistent  with  positive  confirmation  bias. 

Since  the  original  experiments  were  published,  many  replications  and  variations  of  the  same  task  have 
been  conducted  (  Griggs  &  Cox  (1982)  or  Klayman  &  Ha  (1987) ).  These  psychological  experiments 
have  shown  that  humans  often  fail  to  search  for  and  use  negative  information  and  instead  relying  heavily 
on  seeking  out  positive  information. 

This  tendency  has  turned  up  in  e.g.  matching  problems  Donaldson  (1959),  formal  logic  problems 
Anderson  (1985),  and  in  concept  attainment  or  hypothesis  evaluation  tasks  (  Anderson  (1985),  Wason 
(1960) ).  Other  evidence  were  from  further  studies  by  Wason  and  his  collaborators  (  Wason  (1966), 
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Wason  (1968a),  Wason  &  Shapiro  (1971),  Wason  &  Laird  (1972)  using  the  well  known  four-card 
selection  task  to  further  demonstrate  this  tendency. 

The  foremost  question  on  the  mind  of  the  author  was  whether  such  results  are  directly  translatable  even 
though  there  are  ample  studies  on  it.  The  other  concern  that  the  author  had  with  all  these  research  was  that 
they  are  often  conducted  in  non-military  academic  settings.  An  exception  is  Tolcott,  Marvin,  &  Bresoick's 
(1989)  study  of  military  intelligence  analysts.  It  is  one  of  the  rare  examples  of  positive  confirmation  bias 
done  in  a  military  setting.  In  one  part  of  their  studies,  analysts  were  offered  their  choice  to  seek  enemy 
information  that  would  confirm  or  refute  their  initial  hypothesis.  Consistently,  analysts  sought 
information  which  would  confirm  the  hypothesis  rather  than  that  which  would  give  them  the  greatest 
resolution  of  uncertainty. 

The  concern  over  the  translatability  of  the  results  from  past  research  also  stems  from  studies  such  as 
Davidsson  &  Wahlund's  (1992).  Their  study  concluded  that  the  performances  of  the  experimental 
participants  are  extremely  sensitive  to  variations  of  even  the  same  abstract  problem  similar  to  the  original 
Wason  selection  task.  This  therefore  provided  the  impetus  for  dedicated  positive  confirmation  bias 
experimentation  to  be  conducted  in  SCME.  The  results  of  Davidsson  &  Wahlund  (1992)  also  suggested 
that  experimentation  task  specifically  designed  in  a  military  context  should  be  used. 

There  are  also  other  interesting  findings  in  positive  confirmation  bias  research  such  as  in  Jones  &  Sugden 
(2001).  In  their  study,  using  variations  of  the  original  Wason  Selection  Task,  it  was  also  found  that 
learning,  i.e.  experience,  is  not  likely  to  address  the  issue  of  positive  confirmation  bias.  It  is  found  that 
even  people  with  experience  tend  to  seek  out  information  that  confirms  their  tentative  held  hypothesis 
and,  to  a  certain  extent,  other  information  that  has  little  value  in  resolving  uncertainties.  This  provides 
valuable  information  for  the  design  of  future  experiments.  More  importantly,  it  highlights  that  alternate 
hypotheses  generation  is  not  a  function  of  experience  and  there  may  be  times  when  opinions  from  the 
most  inexperienced  crew  counts. 


Research  Questions 

The  review  of  the  positive  confirmation  bias  literature  helped  SCME  nail  down  the  following  focus 
questions  for  the  experiment: 

a.  Can  positive  confirmation  bias  be  addressed  with  no  C2  intervention  in  a  team  setting? 

b.  Is  positive  confirmation  bias  more  acute  when  command  teams  are  put  under  stress  or 
pressure? 

c.  What  kind  of  C2  interventions  can  potentially  address  positive  confirmation  in  a 
command  team  setting? 


Experimental  Methodology 

The  experiments  were  conducted  over  4  days  from  Oct  25  to  28  2004  with  96  graduates  of  the  35th 
Singapore  Command  and  Staff  College  Course.  The  first  two  days  were  used  primarily  for  pilot 
experimentation  as  well  as  to  brainstorm  for  possible  C2  interventions.  The  last  two  days  were  used  for 
formal  experimentation  and  the  results  will  be  discussed  in  detail. 

Each  experimental  unit  consists  of  a  team  of  4  members  for  all  days.  They  were  asked  to  come  to 
individual  decisions  based  on  the  scenarios  presented  before  discussing  and  making  a  team  decision.  With 
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no  C2  interventions  on  the  first  day,  observations  and  analysis  showed  that  team  decisions  were  biased 
especially  in  the  presence  of  dominant  character  or  characters.  Many  team  discussions  were  often  not 
focused  and  went  round  in  circles  despite  given  a  seemingly  a  simple  task  of  deciding  the  sortie  plan  of  an 
Unmanned  Aerial  Vehicle  (UAV).  More  specifically,  the  decision  consists  of  deciding  10  grid  squares  for 
a  UAV  to  perform  an  intelligence  collection  task. 

At  the  end  of  the  brainstorming  session  on  the  first  day,  a  SCSC  graduate  suggested  to  display  the 
differences  in  individual  plans  for  team  discussions.  This  idea  of  a  C2  intervention  was  tested  on  the 
second  day.  The  team  discussions  became  more  focused.  In  fact,  it  became  overly  focused  as  team 
members  merely  discussed  only  differences  in  their  plans  and  often  neglecting  the  considerations  (which 
might  be  different  even  for  similarities  in  plans)  behind  the  similarities  in  their  plans.  This  phenomenon 
led  the  experimentation  team  to  propose  another  intervention  in  which  an  aggregated  version  of  all  the 
individual  plans  was  displayed  so  that  discussions  could  hopefully  focus  on  both  similarities  and 
differences. 

The  following  sections  describe  the  formal  experimentation  carried  out  over  the  last  two  days  with  the 
remaining  48  students. 

Participants 

48  Graduates  of  the  35th  SCSC  participated  in  experiment.  The  participants  were  of  different  background 
and  services.  All  of  them  were  at  the  level  of  at  least  senior  staff  officer  or  executive  officer  level. 
Majority  held  the  major  rank  while  a  small  number  were  lieutenant  colonels.  Their  Myers-Briggs  Type 
Indicator  (MBTI)  scores  were  used  to  distribute  them  evenly  in  teams  of  fours  so  that  no  particular 
personality  types  would  dominate  each  team.  They  were  not  paid  in  token  or  kind  for  the  entire 
experiment. 

Scenario 


Figure  1  shows  a  typical  scenario  (8x8  grid  squares  situation  picture)  presented  during  each  of  the 
experimental  run.  Each  of  the  4  team  member  is  a  battalion  commander  of  an  armor  brigade.  In  the 
narrative,  the  brigade  commander  was  killed  in  action.  The  4  participants  were  told  that  they  were  in 
charged  of  the  brigade  and  were  to  make  all  decisions  concerning  its  operations.  All  the  scenarios 
presented  were  varied  from  a  template.  The  narrative  was  designed  in  such  a  way  to  skew  the  participants 
towards  the  hypothesis  that  blue  had  numerical  superiority  and  the  enemy  tanks  presented  were  the  only 
ones  out  there.  Participants  were  then  told  that  they  had  one  last  UAV  sortie  they  could  execute  to 
determine  whether  to  order  a  commit  or  a  retrograde  operation  based  on  the  outcome  of  this  UAV  sortie. 
For  simplicity,  the  UAV  sortie  plan  consist  of  choosing  10  grid  squares  in  which  the  UAV  would  take  a 
spot  photograph  of  one  grid  square.  Individual  planning  precedes  team  discussion  for  the  final  decision  on 
the  UAV  sortie  plan.  The  outcome  of  the  UAV  intelligence  collection  sortie  was  not  played. 

Apparatus 

A  system  known  as  Missionmate©  (developed  indigenously  by  SCME)  based  on  the  Service-Oriented 
Architecture  (SOA)  was  used  for  the  experiment.  In  the  experimentation  setup  each  participant  was  given 
access  to  a  laptop  to  view  the  situation  picture  as  well  as  to  enter  their  personal  decisions.  The  leader,  a 
fixed  but  arbitrarily  selected  position,  was  asked  to  enter  the  team  decision  in  his  or  her  laptop  at  the  end 
of  the  team  discussion. 
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Any  UAV  search 
grid  that  falls  within 
the  red  circle  is 
given  a  score  of  1. 
Anything  outside  is 
given  a  score  of  -1. 


d  ¥ Ibid 


Figure  1.  Scenario  and  Scoring  Plan 


Procedure 


The  exposure  of  the  participants  to  the  various  combinations  was  randomized.  Participants  turned  up  in 
two  batches  daily  with  about  12  participants  for  each  session.  For  each  session,  the  team  assignments 
were  announced  prior  to  the  start  of  the  experiment.  Teams  were  rotated  in  and  out  of  the  experimentation 
laboratory  (SCME  conference  room)  depending  on  how  quickly  they  completed  their  designated 
experimentation  runs.  The  experiment  took  place  with  the  participants  sitting  around  a  conference  table. 
The  author  and  an  assistant  experimenter  took  the  chairman  positions.  At  the  start  of  each  run,  a 
standardized  verbal  protocol  was  used  to  provide  instructions  on  background,  scenario  information  and 
other  accompanying  instructions.  Individuals  were  then  asked  to  come  to  an  individual  decision  (see 
Figure  2  for  an  example)  followed  by  a  team  decision  through  a  team  discussion. 

Design  of  Experiment 

Two  factors  were  considered:  (1)  Proximity  of  enemy  tanks  and  (2)  Interventions.  Proximity  has  2  levels: 
(1)  near  and  (2)  far.  Interventions  has  3  levels:  (1)  No  Intervention  (2)  Overlap  with  Degrees  (see  Figure 
3)  and  (3)  Aggregate  (see  Figure  4).  A  D-optimal  design  was  used  yielding  24  data  points  on  27  Oct  2004 
and  12  data  points  on  28  Oct  2004. 


Figure  2. 


Figure  3. 


An  Example  of  an  Individual  UAV  Sortie  Plan. 


An  Example  of  the  C2  Intervention  which  shows  all  the  overlap  in  Individual  Plans. 
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Figure  4.  An  Example  of  the  C2  Intervention  which  shows  the  aggregate  of  all  Individual  Plans 

without  Depicting  Differences. 

Measures  of  Performance 


There  were  two  measures  of  performance  used  in  the  experiment.  The  Confirmation  Bias  Index  (CBI) 
adapted  from  Betsch,  Haberstroh,  Glockner,  Haar,  &  Fiedler  (2001)  which  represents  the  aggregate  score 
based  on  the  scoring  plan  assumptions  depicted  in  Figure  1 .  The  normative  team  CBI  score  known  as  the 
Rational  Mid  Point  Team  (RMPT)  CBI  was  calculated  from  the  individual  CBI  scores.  It  assumes  that  the 
logical  output  of  a  team  decision  would  be  the  median  between  the  highest  and  lowest  score  of  the 
individual  CBIs.  The  effectiveness  of  the  interventions  was  determined  by  the  difference  between  the 
eventual  team  CBI  and  the  calculated  RMPT  CBI. 

Hypotheses 

The  hypotheses  of  the  experiments  were: 

a.  Positive  confirmation  bias  exists  among  SAF  commanders. 

b.  Positive  confirmation  bias  is  more  acute  when  decisions  are  made  under  stress. 

c.  Interventions,  i.e.  like  those  in  Figure  3  and  4,  would  result  in  more  negative  CBIs  or 
reduce  positive  confirmation  Bias. 
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Findings  and  Discussions 


Hypothesis  1 :  Existence  of  Confirmation  Bias  in  SAF 
Table  1. 

One -Tail  Simple  T-Test  for  Individual  CBIs. 


N 

M 

t-  statistic 

CBI 

144 

3.78 

5.53* 

Note:  *p<  .05 


A  one-tail  simple  t-test  was  performed  on  the  individual  CB1  scores  to  determine  if  there  is  statistical 
evidence  that  positive  confirmation  bias  exists  among  SAF  commanders.  A  CBI  score  of  more  than  0 
indicates  presence  of  positive  confirmation  bias  while  a  CBI  score  of  less  than  0  does  not. 

From  Table  1,  the  t-statistic  of  5.53  is  significant  at  p  <  0.05.  Although  this  result  may  not  be  surprising  to 
the  reader,  one  has  to  bear  in  mind  the  findings  of  Davidsson  &  Wahlund  (1992)  who  found  that  the 
performance  of  participants  were  very  sensitive  to  even  variations  of  the  original  Wason  selection  task. 
Further,  during  the  course  of  this  research  it  was  also  revealed  that  relatively  few  applied  military 
research  has  been  done  in  the  area  of  addressing  positive  confirmation  bias.  This  piece  of  result  therefore 
allows  SCME  to  set  out  on  a  surer  footing  that  the  interventions  designed  subsequently  is  addressing  an 
existing  problem  rather  than  one  that  may  not  exist. 

The  histogram  plot  of  the  individuals  CBI  provides  further  resolution  of  the  results  in  Table  1 .  A  CBI 
score  of  10  corresponds  to  a  UAV  sortie  plan  where  one  hundred  percent  emphasis  is  on  the  red  targets 
presented  during  each  run.  A  CBI  score  of  -10  corresponds  to  a  UAV  sortie  plan  where  one  hundred 
percent  emphasis  is  on  other  red  targets  that  have  not  been  detected  but  might  be  out  there. 

One  can  see  from  the  histogram  plot  that  a  great  number  of  individual  decisions  (over  40  out  of  the  144) 
tend  to  the  extreme  score  of  10.  The  proportion  of  individual  decisions  tending  to  a  CBI  scores  of  -10  is 
proportionately  lesser.  The  majority  of  the  individual  decisions  fell  in  between  these  two  extremes. 
Invariably,  this  was  the  result  of  two  opposing  hypotheses:  (1)  the  tanks  presented  were  the  only  ones  out 
there  and  (2)  there  are  more  tanks  than  those  presented.  Specifically  for  this  experiment,  a  C2  intervention 
will  be  effective  if  those  with  hypotheses  at  either  end  can  be  made  to  entertain  the  alternate  hypothesis. 
This  result  therefore  provides  first  evidence  that  the  interventions  to  be  tested  in  Hypothesis  No.  3  have  a 
chance  of  working  because  opposing  hypothesis  are  being  held  in  the  teams.  The  question  then  is  to  see 
how  interventions  tested  under  Hypothesis  No.  3  can  be  used  to  flash  out  the  alternate  hypothesis  during 
team  discussions. 

This  is  opposed  to  the  case  where  there  are  no  alternate  hypothesis  alive  among  team  members,  i.e.  we 
will  see  the  spread  of  CBIs  all  above  0  or  vice  versa,  the  design  of  the  C2  intervention  would  have  called 
for  the  need  to  have  an  intelligent  system  that  can  generate  alternate  hypotheses  based  on  the  current 
situation. 


11 


Figure  5.  Histogram  of  Individual  CBI  Landscape. 

Hypothesis  2:  Effects  of  Stress  on  Positive  Confirmation  Bias 

A  one-way  ANOVA  was  performed  to  determine  the  effects  of  proximity  of  threat,  used  as  a  surrogate  for 
stress,  on  the  individual  CBIs.  From  Table  2,  the  F-statistic  of  94.69  is  significant  at  p  <  0.05.  One  could 
also  see  the  profound  effect  of  having  near  threats  and  far  threats  in  Figure  6.  The  mean  CBI  value  is 
above  6  for  near  targets  while  that  for  far  targets  is  just  above  2.  In  terms  of  the  UAV  intelligence 
gathering  emphasis,  it  means  that  80%  versus  20%  of  the  UAV  sortie  was  allocated  to  the  near  targets 
presented  and  60%  versus  40%  of  the  UAV  sortie  was  allocated  for  far  targets.  It  is  not  difficult  to 
postulate  that  in  real  operations  the  effects  of  stress  on  positive  confirmation  bias  is  going  to  be  multiplied 
many  times  over.  Operationally,  friendly  forces  must  therefore  guard  against  this  cognitive  weakness 
while  attempting  to  exploit  a  similar  phenomenon  on  the  adversary  end. 


Table  2. 

ANOVA  Table:  Effects  of  Proximity  on  Individual  CBIs. 


df 

F 

CBI 

1 

94.69* 

error 

142 

(26.23) 

Note:  Values  enclosed  in  parentheses 
represent  mean  square  errors.  *p<  .05 


More  importantly,  this  simple  result  raises  the  question  of  whether  there  are  other  cognitive  weaknesses 
that  could  be  similarly  studied  and  exploited.  This  is  especially  important  when  one  talks  about  concepts 
such  as  Effects-Based  Operations  which  centers  heavily  around  the  exploitation  of  the  adversarial 
cognitive  and  social  domains  from  the  prevalently  used  network  centric  warfare  model  found  in  Alberts, 
Garstka,  Hayes,  &  Signori  (2001) 
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Proximity 


Figure  6.  Effects  of  Proximity  on  Individual  CBI 

Hypothesis  3:  Effects  of  Different  Interventions  on  Team  Positive  Confirmation  Bias 

Table  3. 

ANOVA  Table:  Effects  of  Interventions  on  Distance  Moved  from  RMPT  CBI 


df 

F 

Intervention 

2 

4.37 

Proximity 

1 

1.48 

Intervention*Proximity 

2 

9.30* 

error 

30 

(6.29) 

Note:  Values  enclosed  in  parentheses  represent  mean 
square  errors.  *p<  .25 
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Figure  7.  Effects  of  Different  Interventions  on  Distance  from  RMPT  CBI 

A  two-way  ANOVA  was  performed  to  determine  the  effects  of  proximity  of  threat  and  different 
interventions  had  on  the  difference  vector  between  the  Rational  Mid-Point  Team  CBI  and  the  final  CBI  of 
the  team  decision.  From  Table  3,  the  F-statistic  of  9.30  is  significant  at  p  <  0.25.  While  the  use  of  p  =  0.25 
deviates  from  the  statistical  norms  of  at  least  p  =  0. 10,  the  author  feels  that  even  with  a  25  percent  chance 
of  committing  a  Type  I  error,  the  result  presents  the  merits  for  discussion  because  it  really  points  towards 
a  future  possibility  that  positive  confirmation  bias  could  be  better  controlled  and  mitigated. 

From  the  graph  in  Figure  7,  it  is  clear  that  when  there  are  no  interventions,  i.e.  the  middle  case,  there  were 
no  real  difference  between  the  rational  mid-point  team  CBI  and  the  final  team  decision.  This  augurs  well 
for  this  measure  as  this  was  the  expected  result  under  the  circumstances  where  there  were  no 
interventions.  From  the  same  figure,  the  C2  intervention  where  the  aggregate  plans  were  presented  shows 
the  tendency  to  be  able  to  influence  the  prevalently  positive  CBI  scores  in  the  negative  directions  whether 
it  is  for  near  or  far  targets.  A  surprising  result  was  found  for  the  C2  intervention  that  displayed  the 
differences  of  the  individual  plans.  From  Figure  7,  it  seems  that  it  is  able  to  more  precisely  influence  the 
positive  confirmation  bias  taking  into  account  the  situations  where  targets  are  near  or  far.  Recall  from 
Figure  6  that  near  targets  elicited  extremely  positive  CBI  scores  while  far  targets  elicited  slightly  positive 
ones.  Without  prescribing  any  theoretically  correct  team  CBI  scores,  this  C2  intervention  seems  to  be  able 
to  influence  the  CBI  scores  in  exactly  the  right  direction  assuming  that  a  70%  emphasis  on  presented 
targets  and  30%  emphasis  on  others  not  presented  is  adequate;  i.e.  lowering  the  CBI  scores  for 
presentation  of  near  targets  and  raising  the  CBI  scores  of  far  targets. 

Future  Research 

With  the  preceding  discussions,  the  future  research  in  the  area  of  positive  confirmation  bias  would  centre 
round  answering  the  following  questions: 

a.  Is  there  a  possible  development  for  a  possible  operational  framework  for  exploitation  of 
cognitive  and  social  weaknesses?  Will  this  framework  be  mirrors  of  each  other  when  considering 
friendly  and  adversarial  perspectives? 

b.  What  other  psychological  weaknesses  are  out  there  for  similar  studies  to  be  conducted 
and  C2  interventions  to  be  developed? 
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c.  How  would  the  development  of  deceptive  tactics  be  now  devised  given  the  results  from 
the  examination  of  Hypothesis  2? 

d.  Operational  plans  are  invariably  more  complex.  How  can  simple  interventions  developed 
for  applied  military  research  of  this  kind  be  translated  operationally? 

e.  Are  there  some  other  fundamentally  different  interventions  that  could  be  developed  other 
than  those  tested  in  this  experiment? 


Conclusions 

This  experiment  was  conducted  at  a  very  low  cost  and  is  possibly  one  of  those  few  looking  at  positive 
confirmation  bias  in  a  military  setting  for  both  individuals  and  teams.  Overall  this  whole  exercise  has 
provided  very  useful  insights  that  will  contribute  to  SAF’s  effort  in  sensemaking  as  well  as  future  C2 
systems  design.  What  began  as  a  fuzzy  journey  looking  at  across  the  board  concepts  such  as  groupware 
awareness,  groupthink,  insight/creativity  and  decision  making  biases  turned  out  to  be  an  extremely 
beneficial  and  fruitful  exercise.  Although  it  may  be  some  years  before  an  operational  C2  intervention  of 
this  nature  is  really  developed,  this  research  and  experiment  brings  forth  new  found  confidence  that 
interesting  concepts  can  indeed  be  built  upon  the  fundamental  concept  of  distributed  collaboration. 
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